NAPC: A Neural Algorithm for Automated Passenger Counting in Public Transport on a Privacy-Friendly Dataset

Real-time load information in public transport is of high importance for both passengers and service providers. Neural algorithms have shown a high performance on various object counting tasks and play a continually growing methodological role in developing automated passenger counting systems. However, the publication of public-space video footage is often contradicted by legal and ethical considerations to protect the passengers’ privacy. This work proposes an end-to-end Long Short-Term Memory network with a problem-adapted cost function that learned to count boarding and alighting passengers on a publicly available, comprehensive dataset of approx. 13,000 manually annotated low-resolution 3D LiDAR video recordings (depth information only) from the doorways of a regional train. These depth recordings do not allow the identification of single individuals. For each door opening phase, the trained models predict the correct passenger count (ranging from 0 to 67) in approx. 96% of boarding and alighting, respectively. Repeated training with different training and validation sets confirms the independence of this result from a specific test set.DFG, 414044773, Open Access Publizieren 2021 - 2022 / Technische Universität Berli

Operational Calibration: Debugging Confidence Errors for DNNs in the Field

Trained DNN models are increasingly adopted as integral parts of software systems, but they often perform deficiently in the field. A particularly damaging problem is that DNN models often give false predictions with high confidence, due to the unavoidable slight divergences between operation data and training data. To minimize the loss caused by inaccurate confidence, operational calibration, i.e., calibrating the confidence function of a DNN classifier against its operation domain, becomes a necessary debugging step in the engineering of the whole system. Operational calibration is difficult considering the limited budget of labeling operation data and the weak interpretability of DNN models. We propose a Bayesian approach to operational calibration that gradually corrects the confidence given by the model under calibration with a small number of labeled operation data deliberately selected from a larger set of unlabeled operation data. The approach is made effective and efficient by leveraging the locality of the learned representation of the DNN model and modeling the calibration as Gaussian Process Regression. Comprehensive experiments with various practical datasets and DNN models show that it significantly outperformed alternative methods, and in some difficult tasks it eliminated about 71% to 97% high-confidence (>0.9) errors with only about 10\% of the minimal amount of labeled operation data needed for practical learning techniques to barely work.Comment: Published in the Proceedings of the 28th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2020

Clustering und Prototyp-basierte Klassifikation

Die Arbeit besteht aus zwei Themengebieten, die mehr oder weniger unabhängig voneinander sind. Im ersten Teil dieser Arbeit geht es um die Entwicklung von Algorithmen für einen Nächsten-Prototyp-Klassifikator (engl.: nearest prototype classifier (NPC)). Der zweite Teil behandelt Clustering und Visualisierung von Matrix-Daten. Um einen neuen Datenpunkt zu klassifizieren, muss NPC lediglich die Distanzen zwischen dem Datenpunkt und den Prototypen vergleichen und dem Datenpunkt die Klasse des nächsten Prototyps zuweisen. Weil die Entscheidungsregel sehr einfach und schnell ist, werden NPCs häufig für Echtzeit-Anwendungen wie Spracherkennung verwendet. Learning Vector Quantization (LVQ) ist eine Familie von heuristisch gut gewählten adaptiven Lernalgorithmen für NPC. Der weit verbreitete LVQ 2.1 ist ein Beispiel für einen LVQ Algorithmus, der gute Klassifikationsergebnisse liefert. Da er jedoch heuristisch ist, kann es sein, dass der resultierende NPC nicht optimal ist. Aus diesen Gründen leite ich drei Varianten der LVQ-Methode über einen prinzipielleren Ansatz her. Dabei modelliere ich die Wahrscheinlichkeitsdichte der Daten mittels eines Gauss'schen Mischungs-Modells. Für einen Datenpunkt x und sein Klassenlabel y definiere ich zwei eingeschränkte Wahrscheinlichkeitsdichten, dass x vom Gauss'schen Mischungs-Modell für die Klasse y bzw. die anderen Klassen als y generiert wird. Aus den beiden eingeschränkten Wahrscheinlichkeitsdichten definiere ich drei Kostenfunktionen: eine Fehlklassifikationsrate und zwei Likelihood-Quotienten. Aus den Kostenfunktionen leite ich die Lernalgorithmen für den NPC über eine stochastische Gradientenmethode her. Ich untersuche im weiteren die Eigenschaften der Kostenfunktionen und der Lernalgorithmen und gebe dabei erstmalig eine analytische Begründung für LVQ 2.1. Die vorgeschlagenen Methoden ermöglichen die Erweiterung der LVQ-Familie auf weiche Klassifikation und auf verschiedene Distanzmaße. Bei den Experimenten erzielten die vorgeschlagenen Lernalgorithmen bessere Klassifikationsergebnisse als bisherige LVQ Algorithmen. Ich habe einen neuen Clusteringalgorithmus für paarweise Daten mit Hilfe der Rate-Distortion-Theorie hergeleitet. Ausserdem habe ich die strukturellen Phasenübergänge von dem hergeleiteten Algorithmus und einem Entropie-basierten paarweisen Clusteringalgorithmus mittels einer linearen Stabilitätsanalyse untersucht und dadurch die kritischen Werte des Lagrange-Multipliers berechnet. Um die Konvergenz zu einem lokalen Optimum zu vermeiden, habe ich eine neue Optimierungsmethode, nämlich inkrementelles Splitting für paarweise Clusteringalgorithmen, vorgeschlagen. Für Visualisierungszwecke habe ich die Self-Organizing Map (SOM) Methode auf den hergeleiteten Clusteringalgorithmus und die Information Bottleneck Methode angewandt und so nachbarschaftserhaltende Clusteringalgorithmen für paarweise Daten und Co-ocurrence Daten hergeleitet. Die hergeleiteten neuen Lernalgorithmen und die vorgeschlagene Splitting Methode habe ich auf Protein-, DNA-Micro-Array- und Dokument-Daten angewandt.The thesis contains to topics, which are more or less independent. The first part deals with the development of algorithms for a nearest prototype classifier (NPC). The second part is concerned with clustering and visualization of matrix data. In order to classify a new data point NPC just has to compare the distances between the data point and the prototypes, and to assign the class of the nearest prototype to the data point. Because the decision rule is very simple and fast, NPCs are often used in real-time applications like speech-recognition. Learning Vector Quantization (LVQ) is a family of heuristically well chosen adaptive learning algorithms for NPC. The widely used LVQ 2.1 is an example for an LVQ algorithm which provides good classification performance. Because it is heuristic, the resulting NPC classifier may not be optimal. For these reasons I derive three variants of the LVQ method via a principled approach. To this end, I model the probability density of the data via a Gaussian mixture model. For a data point x and its class label y I define two restricted probability densities, that x was generated by the mixture model for class y or the other classes, respectively. From the restricted probability densities I define three cost functions: a misclassification rate and two likelihood ratios. Using these cost functions, I derive learning algorithms for the NPC via a stochastic gradient method. Moreover, I study the characteristics of the cost functions and the learning algorithms, and for the first time give an analytical explanation for LVQ 2.1. The proposed methods permit the extension of LVQ to different distance measures. In the experiments, the proposed learning algorithms achieved better classification performance than previous LVQ algorithms. I derive a new clustering algorithm for pair-wise data via the rate-distortion theory. Additionally, I analyse the structural phase transitions of the derived algorithm and an entropy based pair-wise clustering algorithm using a linear stability analysis, calculating the critical values of the Lagrangian multiplier. In order to prevent the convergence to a local optimum, I propose a new optimization method, called incremental splitting for pair-wise clustering algorithms. For visualization purposes, I apply the Self-Organizing-Map (SOM) method on the derived clustering algorithm and the Information Bottleneck method, thus deriving neighbourhood-preserving clustering algorithms for pair-wise and co-occurrence data. I apply the derived novel learning algorithms and the proposed splitting method on protein, DNA-micro-array and document data

Self-organizing maps and clustering methods for matrix data. Neural Networks

In this contribution we present extensions of the Self Organizing Map and clustering methods for the categorization and visualization of data which are described by matrices rather than feature vectors. Rows and columns of these matrices correspond to objects which may or may not belong to the same set, and the entries in the matrix describe the relationships between them. The clustering task is formulated as an optimization problem: Model complexity is minimized under the constraint, that the error one makes when reconstructing objects from class information is fixed, usually to a small value. The data is then visualized with help of modified Self Organizing Maps methods, i.e. by constructing a neighborhood preserving non-linear projection into a low-dimensional “mapspace”. Grouping of data objects is done using an improved optimization technique, which combines deterministic annealing with “growing ” techniques. Performance of the new methods is evaluated by applying them to two kinds of matrix data: (i) pairwise data, where row and column objects are from the same set and where matrix elements denote dissimilarity values and (ii) co-occurrence data, where row and column objects are from different sets and where the matrix elements describe how often object pairs occur. key words: matrix data, Self-Organizing Maps, pairwise clustering, information bottleneck, deterministic annealing, growing network, rate distortion theory, maximum entropy methods, cooccurrence data.

Soft Learning Vector Quantization

Learning Vector Quantization is a popular class of adaptive nearest prototype classifiers for multiclass classification, but learning algorithms from this family have so far been proposed on heuristic grounds. Here we take a more principled approach and derive two variants of Learning Vector Quantization using a Gaussian mixture ansatz. We propose an objective function which is based on a likelihood ratio and we derive a learning rule using gradient descent. The new approach provides a way to extend the algorithms of the LVQ family to different distance measure and allows for the design of "soft" Learning Vector Quantization algorithms. Benchmark results show that the new methods lead to better classification performance than LVQ 2.1. An additional benefit of the new method is that model assumptions are made explicit, so that the method can be adapted more easily to dierent kinds of problems

Soft Nearest Prototype Classification

We propose a new method for the construction of nearest prototype classifiers which is based on a Gaussian mixture ansatz and which can be interpreted as an annealed version of Learning Vector Quantization. The algorithm performs a gradient descent on a cost-function minimizing the classification error on the training set. We investigate the properties of the algorithm and assess its performance for several toy data sets and for an optical letter classification task. Results show (i) that annealing in the dispersion parameter of the Gaussian kernels improves classification accuracy, (ii) that classification results are better than those obtained with standard Learning Vector Quantization (LVQ 2.1, LVQ 3) for equal numbers of prototypes and (iii) that annealing of the width parameter improved the classification capability. Additionally, the principled approach provides an explanation of a number of features of the (heuristic) LVQ methods

Berlin-APC: A Privacy-Friendly Dataset for Automated Passenger Counting in Public Transport

This document provides a short technical introduction to the Berlin-APC dataset. The dataset consists of two files, a HDF5 file which contains the image sequences, and a CSV file which contains the labels. The CSV file has three columns: (1) the image sequence name; (2) the number of boarding passengers in that image sequence; (3) the number of alighting passengers in that image sequence. The image sequence names also serve as keys in the HDF5 file. The HDF5 file’s datasets (indexed by the aforementioned sequence names) are float16 arrays of the shape (number of frames, 20, 25), the pixel values range between 0–1, and the frame rate is 10 frames per second